# Install packages if needed (uncomment if necessary)
# install.packages("readr")
# install.packages("tidyverse")
# install.packages("car")
# install.packages("here")
# Load libraries
library(car) # For diagnostic tests
library(patchwork)
library(tidyverse) # For data manipulation and visualizationLecture 07 H testing and simple tests II
Lecture 6 - A Brief review
- Hypotheses
- 1- and 2-sided T tests
- Assumptions of parametric tests
Lecture 7 overview
What we will cover today:
- What are the assumptions again and how do you assess them
- What to do when assumptions fail
- Robust tests
- Rank-based tests
- Permutation tests
Lets work with the Lake Trout data as the weights are pretty cool and the assumptions may or may not hold
This is easily translated into any of the other dataframes you might want to use
lake trout
# the stuff above controls the output and is also set at the top so dont need here
# Load the pine needle data
# Use here() function to specify the path
lt_df <- read_csv("data/lake_trout.csv")
# Examine the first few rows
head(df)
1 function (x, df1, df2, ncp, log = FALSE)
2 {
3 if (missing(ncp))
4 .Call(C_df, x, df1, df2, log)
5 else .Call(C_dnf, x, df1, df2, ncp, log)
6 }
Parametric versus non-parametric tests
T-tests are parametric tests
- Parametric tests:
- specify/assume probability distribution from which parameters came
- Basic assumptions of parametric t-tests:
Random sampling
Normality
Equal variance
No outliers
- Non-parametric tests: no assumption about probability distribution
- Mukasa et al 2021 DOI: 10.4236/ojbm.2021.93081
Assumptions of parametric tests
- If assumptions of parametric test violated, test becomes unreliable
- This is because test statistic may no longer follow distribution
- Most parametric tests robust to mild/moderate violations of below assumptions
Assumptions of parametric tests
Basic assumptions of parametric t-tests:
Random sampling
Normality
Equal variance
No outliers
Random sampling:
samples are randomly collected from populations; part of experimental design
Necessary for sample -> population inference
<>
Assumptions of parametric tests
Basic assumptions of parametric t-tests:
- Normality
- equal variance
- random sampling
- no outliers
- Lets do the above for one lake -
NE 12as if we were going to do a one sample T Test- we need to make a new dataframae with NE 12 data only called
ne12_data - how do you do this?
- we need to make a new dataframae with NE 12 data only called
- Normality: Samples from normally distributed population
- Graphical tests: histograms, dotplots, boxplots, qq-plots
- “Formal” tests: Shapiro-Wilk test - sometimes not useful
Assumptions of parametric tests
Basic assumptions of parametric t-tests:
- Normality
- equal variance
- random sampling
- no outliers
- Lets do the above for one lake -
NE 12as if we were going to do a one sample T Test- we need to make a new dataframae with NE 12 data only called
ne12_data - how do you do this?
- we need to make a new dataframae with NE 12 data only called
- Normality: Samples from normally distributed population
- Graphical tests: histograms, dotplots, boxplots, qq-plots
- “Formal” tests: Shapiro-Wilk test - sometimes not useful
[1] "Null hypothesis is that data is normally distributed"
Shapiro-Wilk normality test
data: ne12_data$length_mm
W = 0.94528, p-value = 1.56e-09
Assumptions of parametric tests
Basic assumptions of parametric t-tests:
- Normality
- equal variance
- random sampling
- no outliers
Equal variance: samples are from populations with similar degree of variability
- Graphical tests: boxplots
- “Formal” tests: F-ratio test
- When samples sizes equal
- Parametric tests most robust to violations of normality
- Less so for equal variation assumptions
length_plot <- ne12_data %>% ggplot(aes(x=lake, y = length_mm)) +geom_boxplot()
mass_plot <- ne12_data %>% ggplot(aes(x=lake, y = mass_g)) +geom_boxplot()
length_plot + mass_plot + plot_layout(ncol=1)Assumptions of parametric tests
- Basic assumptions of parametric t-tests: - Normality
- equal variance
- random sampling
- no outliers
- No outliers: no “extreme” values that are very different from rest of sample
- Graphical tests: boxplots, histograms
- “Formal tests”: Grubb’s test - no one really does this
- Note: outliers a problem for non-parametric tests as well
ne12_histo_plot + ne12_box_plot + plot_layout(ncol = 1)Nonparametric test
- What if T Test assumptions fail?
- Alternative tests, with more relaxed assumptions, are available
- In which case would you use the following tests?
- Welch’s t-test: when distribution normal but variance unequal
- Mann-Whitney-Wilcoxon test: when distribution not normal and/or outliers are present (but both groups should still have similar distributions and ~equal variance)
- Permutation test for two samples: when distribution not normal (but both groups should still have similar distributions and ~equal variance)
< –
ne12_histo_plot+ne12_box_plot + ne12_qq_plot + plot_layout(ncol=1)Assumptions of parametric tests
- QQ-plots: tool for assessing normality
- On x- theoretical quantiles from SND
- On y- ordered sample values
- Deviation from normal can be detected as deviation from straight line
isl_ne12_df <- lt_df %>% filter(lake %in% c("NE 12", "Island Lake"))
ne12_island_box_plot <- isl_ne12_df %>%
ggplot(aes(x=lake, y = mass_g, color=lake)) +
geom_boxplot()+
theme_minimal()
# ne12_island_box_plot
ne12_island_qq_plot <- isl_ne12_df %>%
# filter(lake =="NE 12") %>%
ggplot(aes(sample = mass_g , color=lake)) +
stat_qq() +
stat_qq_line() +
labs( x = "Theoretical Quantiles", y = "Sample Quantiles") +
theme_minimal()
ne12_island_box_plot +ne12_island_qq_plot+plot_layout(guides="collect")Assumptions of parametric tests
- In some cases, data can be mathematically “transformed” to meet assumptions of parametric tests
- this can be done in r and usually involves
- log10 transformations
- square root transformations
- and many others… I will have a description soon
<>
Robust tests
- Welch’s t-test
- common “robust” test for means of two populations
- Robust to violation of equal variance assumption, deals better with unequal sample size
- Parametric test (assumes normal distribution)
- Calculates a t statistic but recalculates df based on samples sizes and s
log_ne12_island_box_plot <- isl_ne12_df %>%
ggplot(aes(x=lake, y = log10(mass_g), color=lake)) +
geom_boxplot()+
theme_minimal()
ne12_island_box_plot +log_ne12_island_box_plot +plot_layout(guides="collect")Robust tests
Lets compare a parametric T-Test to a Welch’s t-test
- T-Test is:
- t.test(y1, y2, var.equal = TRUE, paired = FALSE)
- Welch’s T-Test is:
- t.test(y1, y2, var.equal = FALSE, paired = FALSE)
- T-Test is:
[1] "Standard t-test results for lenght_mm:"
Two Sample t-test
data: mass_g by lake
t = 14.181, df = 330, p-value < 2.2e-16
alternative hypothesis: true difference in means between group Island Lake and group NE 12 is not equal to 0
95 percent confidence interval:
2266.304 2996.360
sample estimates:
mean in group Island Lake mean in group NE 12
3165.0000 533.6677
[1] "Welch's t-test results for lenght_mm:"
Welch Two Sample t-test
data: mass_g by lake
t = 5.1368, df = 9.0578, p-value = 0.0006016
alternative hypothesis: true difference in means between group Island Lake and group NE 12 is not equal to 0
95 percent confidence interval:
1473.676 3788.989
sample estimates:
mean in group Island Lake mean in group NE 12
3165.0000 533.6677
Rank based tests
Rank-based tests: no assumptions about distribution (non-parametric)
Ranks of data: observations assigned ranks, sums (and signs for paired tests) of ranks for groups compared
Mann-Whitney U test common alternative to independent samples t-test
Wilcoxon signed-rank test is alternative to paired t-test
- Assumptions: similar distributions for groups, equal variance
- Less power than parametric tests
- Best when normality assumption can not be met by transformation (weird distribution) or large outliers
<
[1] "Mann-Whitney U test results length:"
Wilcoxon rank sum test with continuity correction
data: mass_g by lake
W = 3205.5, p-value = 9.506e-08
alternative hypothesis: true location shift is not equal to 0
Permutation tests
Permutation tests based on resampling: reshuffling of original data
Resampling allows parameter estimation when distribution unknown, including SEs and CIs of statistics (means, medians)
Common approach is bootstrap: resample sample with replacement many times, recalculate sample stats
Use the
permpackageHo: µA = µB
Ha: µA ≠µB
Calculates the difference ∆ in means between two groups
<!
Permutation tests
- Randomly reshuffle observations between groups (keeping nNE 12=323 and nIsland=10), calculate ∆
- Repeat >1,000 times
- Record proportion of the different means i
- This is equivalent to p-value and can be used in “traditional” H test framework
- For a graphical explanation:
Permutation tests
- In R (using ‘perm’ package):
- Assumptions: both groups have similar distribution; equal variance
library(perm)
# Prepare data for permutation test
ne12_perm_data <- isl_ne12_df %>%
filter(lake == "NE 12") %>%
pull(length_mm)
# Randomly sample exactly 25 observations from NE 12 (set seed for reproducibility)
set.seed(123)
ne12_perm_data <- sample(ne12_perm_data, size = 25, replace = FALSE)
island_perm_data <- isl_ne12_df %>%
filter(lake == "Island Lake") %>%
pull(length_mm)
# Calculate the observed difference in means
observed_diff <- mean(ne12_perm_data, na.rm = TRUE) - mean(island_perm_data, na.rm = TRUE)
# Perform permutation test for difference in means using perm package
permTS(ne12_perm_data, island_perm_data,
alternative = "two.sided",
method = "exact.mc",
control = permControl(nmc = 10000))
Exact Permutation Test Estimated by Monte Carlo
data: GROUP 1 and GROUP 2
p-value = 2e-04
alternative hypothesis: true mean GROUP 1 - mean GROUP 2 is not equal to 0
sample estimates:
mean GROUP 1 - mean GROUP 2
-333.08
p-value estimated from 10000 Monte Carlo replications
99 percent confidence interval on p-value:
0.000000000 0.001059383